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ABSTRACT 

The current standards-based and systemic reform has an 
overarching goal: high academic standards for all students. In achieving the 
goal of "science for all, " the key is the construct of science achievement D 
what K-12 students should know and be able to do in science. This paper 
reviews and analyzes the conceptions of science achievement in major reform 
documents, including those on science content standards (NSES and Project 
2061), performance standards (New Standards), and large Dscale assessment 
frameworks (19.96 NAEP and TIMSS) . The analysis of these documents indicates 
that there is an overall agreement on the conceptions of science achievement 
The documents consistently emphasize high achievement in terms of knowledge 
and abilities in science and technology in personal, social, and historical 
perspectives. Despite the overall agreement, there are also noticeable 
variations among the documents because of different contexts and purposes. 
Based on the synthesis of the conceptions of science achievement, the paper 
presents an aggregated view of science achievement. Science achievement is 
conceived of in terms of science content and science process. The components 
of science content include: (a) concepts and theories in physical, life, and 

earth and space science; (b) science, mathematics,' and technology; (c) 
science in personal and social perspectives; (d) history and nature of 
science; and (e) unifying themes. The components of science process include: 
(a) scientific understanding; (b) scientific investigation; (c) scientific 
communication; and (d) scientific habits of mind. The components of science 
process cut across and intersect with the components of science content. The 
paper considers the implications of the aggregated view of science 
achievement for assessment and equity in large education systems. In large- 
scale assessments, some components of science achievement present challenges 
because it is difficult to operationalize them in concrete . terms , to develop 
standardized procedures, to administer on-demand assessment, or to use 
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multiple forms of assessment (e.g., observations, interviews, and products) 
in addition to written forms. Although equity is emphasized, there are 
tensions and dilemmas in considering equity related to science content 
standards and standards-based assessment. What counts as science and what 
should be taught in school science as presented in the content standards are 
often incompatible with ways of knowing and thinking in diverse cultures. The 
relative equity of standardized forms and alternative forms of assessment is 
under consideration. Major reform documents in science education consistently 
emphasize high achievement for all students. The available knowledge about 
assessment and equity, however, is limited. The difficulties with large-scale 
assessments are conceptual and practical, in terms of how to do the 
assessment within the confines of assessment settings. The difficulties with 
educational equity are ideological and cultural, in terms whose science 
should count and be taught in school science, in addition to practical 
matters of resources and opportunities. Now that science content standards 
are established, efforts should be focused on how to implement standards- 
based assessments and how to ensure access and achievement for all students. 
The alignment of assessment with the content standards, as well as the 
attainment of the standards by all students, are key challenges to standards- 
based and systemic reform in large education systems. (Author) 
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Abstract 



The current standards-based and systemic reform has an overarching goal: high academic 
standards for all students. In achieving the goal of “science for all/’ the key is the construct of 
science achievement — what K-12 students should know and be able to do in science. This paper 
reviews and analyzes the conceptions of science achievement in major reform documents, 
including those on science content standards (NSES and Project 2061), performance standards 
(New Standards), and large-scale assessment frameworks (1996 NAEP and TIMSS). The 
analysis of these documents indicates that there is an overall agreement on the conceptions of 
science achievement. The documents consistently emphasize high achievement in terms of 
knowledge and abilities in science and technology in personal, social, and historical perspectives. 
Despite the overall agreement, there are also noticeable variations among the documents because 
of different contexts and purposes. 

Based on the synthesis of the conceptions of science achievement, the paper presents an 
aggregated view of science achievement. Science achievement is conceived of in terms of 
science content and science process. The components of science content include: (a) concepts 
and theories in physical, life, and earth and space science; (b) science, mathematics,' and 
technology; (c) science in personal and social perspectives; (d) history and nature of science; and 
(e) unifying themes. The components of science process include: (a) scientific understanding; 

(b) scientific investigation; (c) scientific communication; and (d) scientific habits of mind. The 
components of science process cut across and intersect with the components of science content. 

The paper considers the implications of the aggregated view of science achievement for 
assessment and equity in large education systems. In large-scale assessments, some components 
of science achievement present challenges because it is difficult to operationalize them in 
concrete terms, to develop standardized procedures, to administer on-demand assessment, or to 
use multiple forms of assessment (e.g., observations, interviews, and products) in addition to 
written forms. Although equity is emphasized, there are tensions and dilemmas in considering 
equity related to science content standards and standards-based assessment. What counts as 
science and what should be taught in school science as presented in the content standards are 
often incompatible with ways of knowing and thinking in diverse cultures. The relative equity of 
standardized forms and alternative forms of assessment is under consideration. 

Major reform documents in science education consistently emphasize high achievement 
for all students. The available knowledge about assessment and equity, however, is limited. The 
difficulties with large-scale assessments are conceptual and practical, in terms of how to do the 
assessment within the confines of assessment settings. The difficulties with educational equity 
are ideological and cultural, in terms whose science should count and be taught in school 
science, in addition to practical matters of resources and opportunities. Now that science content 
standards are established, efforts should be focused on how to implement standards-based 
assessments and how to ensure access and achievement for all students. The alignment of 
assessment with the content standards, as well as the attainment of the standards by all students, 
are key challenges to standards-based and systemic reform in large education systems. 
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Current Conceptions of Science Achievement and 
Implications for Assessment and Equity in Large Education Systems 

The vision of standards -based and systemic reform currently has an overarching goal: 
high academic standards for all students (McLaughlin, Shepard, & O’Day, 1995; Smith & 

O’Day, 1991). In science education, the goal of “science for all” is emphasized in reform 
documents, including National Science Education Standards [NSES] (National Research 
Council [NRC], 1996), Science for All Americans (American Association for the Advancement 
of Science [AAAS], 1989), Benchmarks for Science Literacy (AAAS, 1993), and Scope, 
Sequence, and Coordination of Secondary School Science (National Science Teachers 
Association [NSTA], 1992, 1995, 1996). These documents focus on defining and specifying 
science content standards. 

Recent developments in science content standards are reflected in the assessment 
frameworks of large-scale projects (Raizen, 1997b), including the 1996 National Assessment of 
Educational Progress [NAEP] (National Assessment Governing Board [NAGB], 1994, 1996), 
the Third International Mathematics and Science Study [TIMSS] (Martin & Kelly, 1996; 
McKnight, Schmidt, & Raizen, 1993; Robitallie et al., 1993), and the New Standards Project 
(National Center on Education and the Economy [NCEE], 1997a, 1997b, 1997c). The key 
feature of these projects is to align the assessment with the new content standards in science. 

Central to current reform in science content standards and assessment is the construct of 
science achievement - what K-12 students should know and be able to do in science. The three 
professional organizations in science education (i.e., AAAS, NRC, and NSTA) and the three 
large-scale science assessment projects (i.e.. New Standards, 1996 NAEP, and TIMSS) present 
varying degrees of commonalities and differences in their views of science achievement for all 
students. Because of these variations, there is a need to develop an aggregated view of science 
achievement based on the synthesis of these major reform documents. There is also a need to 
consider the implications for assessment and equity. This information will benefit policy 
makers, subject matter specialists, assessment experts, and evaluators of standards-based and 
systemic reform in general, as well as those involved in science education in particular. 

This paper presents current conceptions of science achievement and implications for 
assessment and equity in the context of standards-based and systemic reform. The paper has four 
main purposes: 

1. To review and analyze conceptions of science achievement in major reform documents 
and to identify commonalities and differences among these documents; 

2. To describe an aggregated view of science achievement based on the synthesis of the 
conceptions of science achievement in these documents; 

3. To consider the implications of the aggregated view of science achievement for large- 
scale assessments; and 

4. To consider the implications of the aggregated view of science achievement for equity. 



The Context for Science Achievement 



Before addressing the main issues of the paper, it is important to consider these issues in 
the context of standards-based and systemic reform in science education. The following are 
discussed here: (a) the general context for science education reform; (b) the construct of science 
achievement; (c) the relationships among content standards, performance standards, and 
assessment frameworks, and (d) the rationale for including NSES, Project 2061, New Standards, 
1996 NAEP, and TIMSS in this paper. 



The General Context for Science Education Reform 

Current science education reform is motivated by social and economic challenges, as well 
as academic purposes (Raizen, 1997a). Knowledge of science and technology has become 
essential for average citizens as they make decisions about personal and social matters, such as 
health, population growth, natural resources, environment, and safety. In addition, the expanding 
global economy demands a work force that is adequately educated in science and technology. In 
response to these social and economic challenges, science education reform documents define 
what all students should know and be able to do in science in order to participate effectively in 
society. These documents define science content standards in a comprehensive manner that 
includes not only science knowledge and inquiry, but also how science is related to personal, 
social, and historical perspectives. 

The need for equity in the goal of “science for all” is obvious with the increasingly 
diverse student population in the U.S. Traditionally, some groups have not performed well in 
science and have been underrepresented in science-related careers, including students from 
diverse cultures and languages, students with disabilities, students from low socio-economic 
backgrounds, and female students. As the global economy expands, these student groups will 
enter the work force of the future. Thus, the emphasis on the achievement of high academic 
standards for all students, especially those who have traditionally been bypassed in science, is an 
important contribution of the reform documents. However, the concept of equity is not clearly 
articulated, and the plan to achieve equity is not well established (Atwater, 1994; Lee & Fradd, 
1998). 



Assessment plays a central role in determining the extent to which science education 
reform has achieved both high academic standards and educational equity (McLaughlin, 

Shepard, & O’Day, 1995; Smith & O’Day, 1991. Particularly, large-scale assessments at the 
state and national levels (e.g., NAEP and TIMSS) are used for accountability purposes as the 
general public and policy makers use these assessments to determine the effectiveness of reform 
efforts in meeting the standards and to monitor the progress in closing achievement gaps among 
diverse student groups. To be effective indicators of the reform, assessments must be aligned 
with content standards and be fair with all students. 

Content standards, equity, and assessment are major areas of emphasis in standards-based 
and systemic reform. Standards-based reform aims to foster high achievement of all students by 
improving the education system at all levels (McLaughlin, Shepard & O’Day, 1995; Smith & 
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O’Day, 1991). Systemic reform is an initiative to change the education system by addressing 
major components of the system together rather than individually (Cohen, 1996; Elmore, 1996). 
The ultimate goal of systemic reform is high achievement for all students, along with valid and 
fair assessment of student achievement. Because of such a close connection between the 
concepts of standards-based reform and systemic reform toward the common goal of high 
achievement for all students, this paper uses the expression ‘‘standards-based and systemic 
reform” in the context of large education systems. 

The Construct of Science Achievement 

The construct of “science achievement ,” although deceptively simple and elusive, 
represents a great challenge: 

Measuring educational achievement is difficult from both a conceptual and a 
practical perspective. What counts as “achievement” is not always easy to 
discern. Even when a concept of achievement has been clearly explicated, ways 
and means for assessing it are not easily devised. (Robitallie et al., 1993, p. 36) 

The construct of science achievement can be interpreted according to the 
conceptual framework of intended, implemented, and attained curriculum in 
TIMSS (Robitallie et al., 1993, pp. 25-30). 

First, the intended curriculum is the science content as defined at the national or 
the educational system level. The intended curriculum may be described in terms of 
science concepts, processes, and attitudes that students are expected to acquire. Second, 
the implemented curriculum is the science content as it is interpreted by teachers in their 
interactions with students. The implemented curriculum is influenced by the intended 
curriculum and can be described in terms of science concepts, processes, and attitudes. 

Finally, the attained curriculum consists of the outcomes of learning — science concepts, 
processes, and attitudes that students have acquired. 

According to this conceptual framework, the intended curriculum indicates science 
content standards at the educational system level; the implemented curriculum refers to science 
instruction at the classroom level; and the attained curriculum represents science achievement at 
the student level. At all three levels, what counts as science achievement (e.g., science concepts, 
processes, and attitudes in TIMSS) must be explicated. Then, ways to assess science 
achievement need to be established. 

Content Standards, Performance Standards, and Assessment Frameworks 

Content standards are broad and general statements of expected learning outcomes — 
“Broad descriptions of the knowledge, skills, and understandings that schools should teach and 
students should acquire in a particular subject area” (McLaughlin, Shepard, & O’Day, 1995, p. 
69). Content standards (e.g., NSES) can be further developed as benchmarks in terms of more 
specific learning outcomes at various grades or grade clusters (e.g.. Benchmarks for Science 
Literacy). 
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Performance standards extend the content standards and benchmarks by specifying “how 
good is good enough” in learning outcomes (e.g., New Standards Project). Performance 
standards include “concrete examples and explicit definitions of what students should know and 
able to do to demonstrate proficiency in the skills, knowledge, and understanding framed by the 
content standards” (McLaughlin, Shepard, & O’Day, 1995, p. 70). 

Assessments provide “an operational definition of standards, in that they define in 
measurable terms what . . . students should learn” (NRC, 1996, p. 6). Based on content 
standards, assessment frameworks identify expectations of learning outcomes to be assessed 
(e.g., NAEP and TIMSS). Assessment frameworks are used as blueprints for assessment 
activities. In this sense, assessment frameworks represents an interim step in translating content 
standards into assessment activities. The complexities and difficulties in aligning assessment 
with content standards are described later in the section on assessment. 

Selection of Major Reform Documents in Science Education 

This paper examines current conceptions of science achievement based on major reform 
documents in science education. These documents are: (a) two sets of documents on content 
standards, including NSES (NRC, 1996) and Project 2061 (AAAS, 1989, 1993); (b) a set of 
documents on performance standards in the New Standards Project (NCEE, 1997a, 1997b, 
1997c); and (c) two sets of documents on assessment frameworks, including 1996 NAEP 
(NAGB, 1994, 1996) and TIMSS (Martin & Kelly, 1996; McKnight, Schmidt, & Raizen, 1993; 
Robitallie et al., 1993). 

These reform documents are selected based on the following criteria. First, they provide 
guidelines for standards-based and systemic reform in large education systems. Second, they 
present comprehensive views of science and science education. Third, they cover science 
content for all grade levels, K-12. Finally, they are key documents that are representative of 
content standards, performance standards, and assessment frameworks in science education. 

Based on the above criteria, the paper does not include some noteworthy efforts. For 
example. National Science Teachers Association documents (1992, 1995, 1996) are limited to 
the four science disciplines (biology, chemistry, earth and space science, and physics) 
traditionally studied at the secondary school level, grades 6-12, The Advanced Placement tests 
are administered to a small population of advanced high school students. The National 
Educational Longitudinal Study (NELS) focused only on 8th grade students before the current 
systemic reform in science had taken hold. 

Conceptions of Science Achievement in Major Reform Documents: 
Commonalities and Differences 

The five sets of documents all present views of science achievement in the context of 
current standards-based and system reform. In this section, commonalities and differences in the 
views of science achievement among these documents are analyzed. This analysis will provide 
the basis for developing a synthesis of current conceptions of science achievement, to be 
described in the next section. 
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For each set of documents, brief descriptions about its background, purposes, and the 
conception of science achievement are presented. The summary of the conceptions of science 
achievement in these documents is shown in Table 1. This table is organized using the 
categories given in the NSES standards because this document represents an effort to establish a 
general consensus of the science and science education communities in the nation. The table 
provides an overview of commonalities and differences among the five sets of documents at the 
categorical or topical level (see Raizen, 1997b, for general descriptions about these documents). 

Several points need to be made clear about Table 1 and the discussion here. First, the 
sequence of the eight categories in NSES is slightly changed to fit with the other four sets of 
documents in Table 1. Second, in addition to the categories of “content standards” in NSES, the 
table includes “process standards” that cut across content standards. This distinction is based on 
“content” and “process” standards in mathematics standards (National Council of Teachers of 
Mathematics, 1989; also see AAAS, 1993, p. 209; Romberg, 1997), even though the authors did 
not state the distinction. New Standards, NAEP, and TIMSS identify science process standards. 
Although NSES and Project 2061 do not identify them as such, the documents emphasize 
process standards throughout the texts. Finally, the categories and terms in Table 1 are actual 
descriptors and expressions used in these documents. The documents sometimes use the same 
terms with different meanings, and different terms with similar meanings (as is explained in the 
next section). 
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Table 1 



Conceptions of Science Achievement: Categories of Content and Process Standards from Major 
Reform Documents 





NSES (1996) 


Project 2061 (1989, 
1993) 


New Standards (1997) 


NAEP (1994, 1996) 




Content 

Standards 


Physical science 


The physical setting 


Physical sciences concepts 


Physical science 


Ph 


Earth/ space 
science 


The physical setting 


Earth and space sciences 
concepts 


Earth science 


Ea 


Life science 


The living 
environment 
The human 
organism 


Life sciences concepts 


Life science 


Lil 




The nature of 
mathematics 
The mathematical 
world 






Sci 


Science and 
technology 


The nature of 
technology 
The designed world 


The designed world 
Impact of technology 


The nature of science and 
technology 


Sci 


Science in personal 
and social 
perspectives 


Human society 


Health, environment, safety, 
resources 




En 


Science as inquiry 


The nature of 
science 

Habits of mind 


Scientific investigation 


Scientific investigation 
The nature of science 


Im 

Nc 


History and nature 
of science 


Historical 
perspectives 
The nature of 
science 


Science as a human 
endeavor 
Impact of science 


The nature of science 


Hi 

Nc 


Unifying concepts 
and processes 


Common themes 


Big ideas and unifying 
concepts 


Themes 





BEST COPY AVAIlABIiS 
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Table 1 (Continued) 



Category 


NRC (1996)» 


Project 2061 
(1989, 1993)» 


New Standards (1997) 


NAEP (1994, 1996) 










Conceptual understanding 


Conceptual understanding 


U 








Scientific investigation 


Scientific investigation 


In 


Process 

Standards 






Scientific tools and technologies 




U 








Scientific thinking 


Practical reasoning 


T1 








Scientific communication 




C 



* Although the NSES and Project 2061 documents do not identify process standards, they 
emphasize these standards 
throughout the texts. 
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National Science Education Standards 



NSES (NRC, 1996) presents “a vision of science education that will make scientific 
literacy for all a reality” (p. ix) and provides a roadmap for how to achieve the goal. The 
development of the NSES document was guided by the following principles: (a) science is for all 
students; (b) learning science is an active process; (c) school science reflects the intellectual and 
cultural traditions that characterize the practice of contemporary science, and (d) improving 
science education is part of systemic education reform (pp. 19-21). The core of NSES involves 
science content standards, while “the separate standards for assessment, teaching, program, and 
system describe the conditions necessary to achieve the goal of scientific literacy for all students 
described in the content standards” (p. 13). 

The content standards define “what students should know, understand, and be able to do 
in natural science” over the course of K-12 science education (p. 103). These standards indicate 
expectations for students’ learning outcomes. While emphasizing “scientific inquiry is at the 
heart of science and science learning” (p. 15), the document presents eight categories of science 
content standards for grade ranges K-4, 5-8, and 9-12: 

• Unifying concepts and processes 

• Science as inquiry 

• Physical science 

• Life science 

• Earth and space science 

• Science and technology 

• Science in personal and social perspectives 

• History and nature of science 

Project 2061 

Science for All Americans (AAAS, 1989) is a major milestone in shaping the discourse of 
science education reform since late 1980s. This document provides a definition of scientific 
literacy for all students by the 12th grade in order to become educated citizens in society. 
Subsequently, Benchmarks for Science Literacy (AAAS, 1993) specifies the components of 
science content in greater detail at different grade levels, including 855 benchmarks at grade 
ranges K-2, 3-5, 6-8, and 9-12. Because of its specificity of learning outcomes, the Benchmarks 
document has often been used as guidelines for K-12 science curriculum. 

Project 2061 defines science broadly to include natural science, mathematics, technology, 
and social science. The science content is organized thematically under four major dimensions 
(see the list below) (AAAS, 1989, p. ix). Then, the categories of the science content are outlined 
in terms of what students should know and be able to do as members of a scientifically literate 
society. Project 2061 highlights “both scientific knowledge of the world and scientific habits of 
mind” as fundamental in science and science learning (AAAS, 1989, p. 190). 

The nature of science, mathematics, and technology 

• The nature of science 
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• The nature of mathematics 

• The nature of technology 

Knowledge and skills in science 

• The physical setting (physical and earth science) 

• The living environment (life science) 

• The human organism (life science) 

Knowledge and skills in related disciplines 

• Human society 

• The designed world 

• The mathematical world 

Perspectives on science 

• Historical perspectives 

• Common themes 

• Habits of mind 

New Standards 

The New Standards Project is designed to build an assessment system for school districts 
and states “to measure students’ progress toward meeting national standards at levels that are 
internationally benchmarked” (NCEE, 1997a, 1997b, 1997c, p. 2). New Standards focuses on 
specifying performance standards and developing performance measures that teachers can use in 
English language arts, mathematics, science, and applied learning. 

The New Standards assessment system has three interrelated components: performance 
standards, on-demand examinations, and a system of portfolio assessment. In science, by 1997, 
only performance standards have been developed. While content standards specify “what 
students should know and be able to do,” performance standards go to the next level by 
specifying “how good is good enough” in attaining the content standards (p. 3). Performance 
standards translate content standards in a form so that assessment activities can be prepared. 

The performance standards are derived directly from the science content standards in 
NRC (1996) and Project 2061 (AAAS, 1993) and, to some extent, international documents 
including TIMSS. The eight categories of performance standards in science are in two 
dimensions: (a) conceptual understanding that generally reflect the content standards in NSES 
and Project 2061 standards, and (b) “areas that need particular attention and a new or renewed 
emphasis,” (NCEE, 1997a, p. 130), which represent important aspects of scientific inquiry as 
defined in NSES and Project 2061. New Standards include eight categories of performance 
standards in these two dimensions at the levels of elementary school (by the end of 4th grade), 
middle school (by the end of 8th grade), and high school (by the end of 10th grade): 

Conceptual understanding 

• Physical sciences concepts 

• Life sciences concepts 

• Earth and space sciences concepts 

• Scientific connections and applications 
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- Big ideas and unifying concepts 

- The designed world 

- Health, environment, safety, resources 

- Science as a human endeavor 

- Historical and contemporary impact of science 

Areas for particular attention 

• Scientific thinking — Scientific inquiry and problem solving by using 

thoughtful questioning and reasoning strategies, common sense and 

conceptual understanding, and appropriate methods of investigation 

• Scientific tools and technologies — Use of tools and technologies to 

collect and analyze data 

• Scientific communication — Effective scientific communication by clearly 

describing aspects of the natural world 

• Scientific investigation — Projects drawn from the following kinds of 

investigation, including experiments, fieldwork, design, and secondary 

research 

National Assessment of Educational Progress 

NAEP has been the only national-level assessment in various subject areas since its 
inception in 1969. The NAEP reports provide descriptive information about student achievement 
in subject areas, including science, for a national sample at grades 4, 8, and 12. The reports also 
provide group comparisons in terms of ethnicity, gender, and other demographic variables. The 
reports describe the relationships between achievement and certain background variables, such as 
time spent on homework and parent’s educational levels. In addition to the national sample data, 
NAEP reports have provided voluntary state-by-state results since 1990. 

Because of its prominence in assessment at the national and state levels, NAEP has tried 
to reflect changes in curriculum and emerging notions of teaching and learning, while 
maintaining the continuity of information that has been gathered in a long-term trend design 
(Glaser & Linn, 1997). The balance between change and continuity presents complicated 
questions (Jones, 1996). Along with the recent development of content standards, “the 1996 
NAEP Science Achievement attempts to reflect a comprehensive, contemporary view of science 
so that those affected by the National Assessment are satisfied that it addresses the complex 
issues in science education without oversimplification” (NAGB, 1996, pp. 2-3). The 1996 
NAEP science assessment is regarded as the best available means for determining the extent to 
which students across the nation and in each state achieved science content standards (Glaser & 
Linn, 1997; O’Sullivan, Reese, & Mazzeo, 1997). 

In developing the assessment framework, 1996 NAEP considered Project 2061, NSTA, 
and TIMSS among its major sources. The framework has four main dimensions, each with sub- 
dimensions, as summarized below: 
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Fields of Science 

• Earth science 

• Physical science 

• Life science 

Knowing and Doing 

• Conceptual understanding — the ability to understand basic concepts and 

tools used in the process of a scientific investigation 

• Scientific investigation — the ability to use the appropriate tools and 

thinking processes in the doing of science 

• Practical reasoning — the ability to apply scientific knowledge to solve 

everyday problems 

The Nature of Science 

- The historical development of science and technology 

- The habits of mind that characterize science and technology 

- The methods of inquiry and problem solving 

Themes 

Third International Mathematics and Science Study 

This is the largest study of mathematics and science performance ever undertaken, 
involving half a million students from 41 countries. The study provides information about 
student performance in mathematics and science for each country, as well as comparisons of 
performance among the countries. The study also provides information about contextual 
variables associated with student performance. These variables include curriculum analyses 
based on national or regional curriculum guidelines and commonly used textbooks, instructional 
practices, perceptions of teachers and principals, and instructional environments. 

TIMSS developed curriculum frameworks in mathematics and science which were used 
for developing achievement tests for ages 9, 13, and the final year of secondary education 
(Martin & Kelly, 1996; Robitallie et al., 1993). Because TIMSS is an international study, the 
frameworks were designed to reflect the curricula of the participating countries. The frameworks 
have three main aspects: (a) subject matter content; (b) performance expectations (i.e., the kinds 
of performances that students are expected to demonstrate while engaged with the content); and 
(c) perspectives (i.e., attitudinal and motivational factors). The components of the content and 
the performance expectations aspects in science achievement tests include the following: 

Content aspect 

• Earth sciences 

• Life sciences 

• Physical sciences 

• Science, technology, and mathematics 
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• History of science and technology 

• Environmental issues 

• Nature of science 

Performance expectations aspect 

• Understanding 

• Theorizing, analyzing, and solving problems 

• Using tools, routine procedures, and science processes 

• Investigating the natural world 

• Communicating 

Analysis of the Documents: Commonalities and Differences 

There is a general agreement on major categories of science content standards among the 
five sets of documents, as summarized in Table 1. There are also differences in some of the 
categories. Beyond this categorical analysis, there are both commonalities and differences in 
underlying views of science achievement among these documents, to be discussed next. 

Commonalities. According to the analysis conducted by the Project 2061, there is about 
90% agreement in content standards between the NSES and Project 2061 documents (AAAS, 
1996, 1997). NSES also states that, “use of Benchmarks ... complies fully with the spirit of the 
content standards [in the NSES]” (NRC, 1996, p. 15). The performance standards in New 
Standards are “built directly upon the consensus content standards,” (NCEE, 1997a, p. 3), 
particularly NSES and Benchmarks for Science Literacy (p. 130). The assessment frameworks 
by 1996 NAEP and TIMSS also reflect the recent developments of science content standards. 

Together, these documents define science achievement in a comprehensive manner. 
Science achievement, i.e., what K-12 students should know and be able to do, includes concepts 
and theories in physical, life, and earth and space sciences; scientific inquiry or investigation; 
science with mathematics and technology; science in personal and social perspectives; the nature 
and history of science; unifying concepts or common themes; and scientific habits of mind. In 
addition, these documents define science achievement at different developmental levels (i.e. 
generally at elementary, middle, and high school years) and at certain depths of knowledge and 
abilities for all students. 

Differences. The documents present noticeable differences in conceptions of science 
achievement. Main differences are described in three ways: (a) differences between the two sets 
of content standards; (b) differences between the two sets of content standards and the three 
assessment frameworks; and (c) differences among the three assessment frameworks. 

Despite an overall agreement, there are noticeable differences between the NSES and 
Project 2061 documents (see AAAS, 1996, 1997 for more detailed discussion). First, Project 
2061 defines science broadly to include natural science, mathematics, technology, and social 
sciences, whereas NSES focuses on natural science. The omission of mathematics in NSES has 
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been under debate, especially considering that NSES emphasizes scientific inquiry and that 
mathematics is integral in scientific inquiry. 

Second, NSES views scientific inquiry as central in science and science learning, whereas 
Project 2061 emphasizes scientific knowledge and habits of mind. Scientific inquiry plays a 
special role in NSES. Scientific inquiry is both an overarching goal basic to science education 
and one of the eight categories of content standards. As an overarching goal, through scientific 
inquiry, students develop the knowledge and abilities for the other categories of content 
standards (NRC, 1996, p. 105). As a category of content standards, scientific inquiry includes 
both “the ability to conduct inquiry” (process) and “understanding about scientific inquiry” 
(content) (p. 105). In contrast. Project 2061 emphasizes scientific knowledge and understanding 
of core concepts and theories. Project 2061 also stresses scientific habits of mind, which are 
defined as certain values, attitudes, and skills shared in the science community. 

Third, there is no clear comparison between NSES standards on “science as inquiry” and 
“history and nature of science” and Project 2061 standards on “the nature of science,” “historical 
perspectives,” and “habits of mind” (see Table 1). There is a significant overlap among these 
standards, and they are sometimes used interchangeably. As students engage in scientific inquiry, 
they come to understand the nature of science and the historical development of science (content) 
and also develop scientific habits of mind (process) (AAAS, 1993, p. 209; NRC, 1996, pp. 45- 
46, 50-51). 

Several main differences are noted between the two sets of content standards and the 
three assessment frameworks. First, NSES and Project 2061 include science content primarily, 
although NSES stresses scientific inquiry and Project 2061 stresses scientific habits of mind as 
science process. In contrast to NSES and Project 2061, the three assessment frameworks 
emphasize science content and process as equally important. New Standards adds four 
categories of standards related to science process, including scientific thinking, tools and 
technologies, scientific communication, and scientific investigation. These process standards 
complement the conceptual understanding standards derived from the content standards in NSES 
and Project 2061. The 1996 NAEP framework identifies three elements of “knowing and doing 
science” (conceptual understanding, scientific investigation, and practical reasoning) that cut 
across three “fields of science” (earth, physical, and life science). TIMSS includes the 
performance expectations aspect, in addition to the content aspect. Further, the 1996 NAEP and 
TIMSS assessment frameworks use the matrix of content-process intersections. 

Second, there is a difference in the balance of representations or emphases among content 
standards. NSES and Project 2061 documents give equal importance to all categories of content 
standards, and do not differentiate the priority of one standard over another (Webb, 1997, pp. 20- 
21). NSES states that “None of the eight categories of content standards should be eliminated . . 

. No standards should be eliminated from a category” (NRC, 1996, pp. 11-112). In contrast, 
three assessment frameworks distinguish the importance by assigning differential weights among 
categories of content standards. For example, in terms of the number of assessment items, 
assessment time, and maximum score points, 1996 NAEP (NAGB, 1994, 1996; O’Sullivan, 
Reese, & Mazzeo, 1997) and TIMSS (Martin & Kelly, 1996; McKnight, Schmidt, & Raizen, 
1993) give more emphases on life, physical, and earth and space science than the rest of the 
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categories of content standards. New Standards maintains the categories of physical science, life 
science, and earth and space science, but combines the four categories in NSES (science and 
technology, science in personal and social perspectives, history and nature of science, and 
unifying concepts and process) into one category called “scientific connections and 
applications.” 

Differences among the three assessment frameworks, shown in Table 1, reside partially in 
the purposes and contexts of the documents. The four categories of science process in New 
Standards respond to the NSES emphasis on scientific inquiry, and New Standards specifies 
performance measures for scientific inquiry (NCEE, 1997a, p. 130). Because NAEP tracks the 
progress in science achievement of U.S. students, it has to maintain the continuity of the 
assessment framework over the years, while representing the changes at certain times. TIMSS 
occurred in an international context and had to consider the curricula of all participating 
countries. 



Science Achievement: Synthesis of Current Conceptions 

Based on the analysis of commonalities and differences in the views of science 
achievement among the five sets of documents, an aggregated view of science achievement is 
presented in this section. The summary is presented in Table 2. Five components of science 
content and four components of science process emerge from the analysis of these documents. 
For each component, key indicators are identified, rendering the component more specific and 
concrete. In this sense, these indicators serve as the operational definition of each component. 
In addition, similarities and differences in key terms and their meanings across the documents 
are clarified. Examples of the components and indicators are provided from emerging research 
and literature in science education. 
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Table 2 



Science Achievement: An Aggregated View from Major Reform Documents 





Components 


Indicators 


Document Sources 




Physical Science 
Life Science 

Earth and Space Science 


• key concepts and theories 

• key vocabulary 


All 




Science, Mathematics, and 
Technology 








Mathematics 


• measurement concepts 

• statistics and probability 
concepts 


Project 2061, TIMSS 




Technology 


• engineering and design 


All 


Content 


Personal and Social Perspectives 


• health 

• population growth 

• natural resources 

• environmental quality 

• safety and hazards 


NSES, Project 2061, 
TIMSS, New 
Standards 




History and Nature of Science 








History of science 


• historical developments of 
major discoveries 

• contributions of diverse cultures 


All 




Nature of science 


• nature of scientific knowledge 

• nature of scientific inquiry 

• the scientific world view 


All 




Unifying Themes 


• systems 

• models 

• constancy and change 

• evolution and equilibrium | 

• form and function 


NSES, Project 2061, 
NAEP, New 
Standards 




Scientific Understanding 


• key concepts and theories 

• relationships among concepts 
and theories 

• explanations of natural 

phenomena 

• applications to new situations 


All 


Process 


Scientific Investigation 


• a systemic observation, a fair 

test, or a controlled experiment 

• scientific tools and equipment 


All 




Scientific Communication 


• multiple representations 

• rules of the discourse of science 


NSES, Project 2061, 
TIMSS, New 
Standards 




Scientific Habits of Mind 


• values and attitudes 


NSES, Project 2061 
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• thinking skills 
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Science Content 



The components of science content are generally consistent with the NSES standards (see 
Table 1). The five components include: (a) physical, life, and earth and space sciences; (b) 
science, mathematics, and technology; (c) science in personal and social perspectives; (d) history 
and nature of science; and (e) unifying themes. Each of these components is described below. 

Physical, life, and earth and space sciences. All of the documents identify three fields or 
disciplines of science. NSES defines this category of standards as follows: “Science subject 
matter focuses on the science facts, concepts, principles, theories, and models that are important 
for all students to know, understand, and use” (p. 106). Main science topics from these 
documents are presented in Table 3. 
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Table 3 



Topics in Physical, Life, and Earth and Space Sciences from Major Reform Documents 



Fields 

of 

Science 


NSES (1996) 


Project 2061 
(1989, 1993) 


New Standards (1997) 


NAEP (1994, 1996) 




K-4 


K-2. 3-5, 6-8, 9-12 


Elementary school (4th) 


4th, 8th, 12th 


Earth 


Properties of earth 


The universe 


Properties of earth 


Solid earth 


and 


materials 


The earth 


materials 


(lithosphere) 


Space 


Objects in the sky 


Processes that 


Objects in the sky 


Water (hydrosphere) 


Science 


Changes in the earth 


shape the earth 


Changes in the earth and 


Air (atmosphere) 




and sky 




sky 


Earth in space 








Middle school (8th) 






Structure of the earth 




Structure of the earth 






system 




system 






Earth's history 




Earth's history 






Earth in the solar 




Earth in the solar system 






system 




Natural resource 










management 






9-12 










Energy in the earth 




High school (10th) 






system 




Energy in the earth system 






Geochemical cycles 




Geochemical cycles 






Origin and evolution 




Origin and evolution of 






of the earth system 




the earth system 






Origin and evolution 




Origin and evolution of 






of the universe 




the universe 










Natural resource 










management 





BEST COPT 



18 




27 



Table 3 (Continued) 



Fields 

of 

Science 


NRC (1996) 


Project 2061 
(1989, 1993) 


New Standards (1997) 


NAEP (1994, 1996) 






K-4 


K-2, 3-5, 6-8, 9-12 


Elementarv school f4thl 


4th, 8th, 12th 




Life 


Characteristics of 


Diversity of life 


Characteristics of 


Changes and 




Science 


organisms 


Heredity 


organisms 


evolution 






Life cycles of 


Cells 


Life cycles of organisms 


Cells and functions 


Di 




organisms 


Interdependence of 


Organisms and 


Organisms 






Organisms and 


life 


environments 


Ecology 






environments 


Flow of matter and 


Change over time 










energy 






Lit 




5i8 


Evolution of life 


Middle school fSth) 








Structure and 


Human identity 


Structure and function in 








function in living 


Human 


living systems 




Lit 




systems 


development 


Reproduction and 








Reproduction and 


Basic functions 


heredity 








heredity 


Learning 


Regulation and behavior 




Int 




Regulation and 


Physical health 


Population and 








behavior 


Mental health 


ecosystems 




Hi 




Population and 




Evolution, diversity, and 








ecosystems 




adaptation of 








Diversity and 




organisms 








adaptations of 












organisms 




High school flOthI 












The cell 








9-12 




Molecular basis of 








The cell 




heredity 








Molecular basis of 




Biological evolution 








heredity 




Interdependence of 








Biological evolution 




organisms 








Interdependence of 




Matter, energy, and 








organisms 




organization in living 








Matter, energy, and 




systems 








the organization in 




Behavior of organisms 








living systems 












Behavior of 












organisms 
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Table 3 (Continued) 



Fields 


NSES (1996) 


Project 2061 


New Standards (1997) 


NAEP (1994, 1996) 




of 

Science 




(1989, 1993) 










K-4 


K-2, 3-5, 6-8, 9-12 


Elementary school (4th) 


4th, 8th, 12th 




Physical 


Properties of objects 


Structure of matter 


Properties of objects and 


Matter and its 


Science 


and materials 


Energy and 


materials 


transformations 






Position and motion 


transformations 


Potion and motion of 


Energy and its 


Mi 




of objects 


Motion 


objects 


transformations 


Str 




Light, heat, electricity. 


Forces of nature 


Light, heat, electricity, and 


Motion 


En 




and magnetism 




magnetism 




Ph 




5i8 




Middle school (8th) 








Properties and 




Properties and changes of 




Ch 




changes of 




properties in matter 








properties in 




Motions and forces 




Fo 




matter 




Transfer of energy 








Motions and forces 
Transfer of energy 




High school (10th) 
Structure of atoms 








9-12 




Structure and properties 








Structure of atoms 




of matter 








Structure and 




Chemical reactions 








properties of 




Motions and forces 








matter 




Conservation of energy 








Chemical reactions 




and increase in 








Motions and forces 




disorder 








Conservation of 




Interactions of energy and 








energy and 




matter 








increase in 












disorder 












Interactions of energy 












and matter 











20 




29 



Overall consistency exists among the documents for the topics of physical science, 
including matter, energy, and motions and forces. Although there seems to be some consistency 
for the topics of earth and space science, including the earth, the solar system, and the universe, 
the consistency for the topics of life science is less clear. The analysis by topics may be too 
general to indicate the degree of agreement among these documents. The same topics may 
include different content, whereas different topics may include similar content. Also, the same 
topics may be interpreted in different ways at different levels of specificity and at different grade 
levels. 



There is an overall agreement on science topics and content among NSES, Project 2061, 
and New Standards (AAAS, 1996, 1997; NCEE, 1997a, 1997b, 1997c). The 1996 NAEP and 
TIMSS frameworks were designed to assess “common denominators” of science topics and 
content across the states in the U.S. (NAEP) or the participating countries (TIMSS). A more 
specific analysis of the correspondence among the five sets of documents is beyond the scope of 
this paper. 

Within the context of standards-based and systemic reform, there is a general consensus 
on the topics and content of physical, life, and earth and space science. U.S. science education 
has been criticized for being “a mile wide and an inch deep,” indicating that the curriculum 
covers too many topics superficially and does not allow students sufficient time to develop a 
deep understanding (Schmidt, McKnight, & Raizen, 1997). In the current reform, the focus is on 
a small number of key topics at greater depth — the principle of “less is more.” 

Science vocabulary has traditionally played an important role in science. Science 
textbooks are often filled with technical terms, sometimes in bold type to capture the attention of 
readers. Project 2061 emphasizes key vocabulary words to the extent that they enhance the 
precision and sophistication of understanding, while strongly discouraging the glib use of these 
words without adequate understanding (e.g., AAAS, 1989, pp. xiv, p. 190; AAAS, 1993, pp. 263, 
312): 



Understanding rather than vocabulary should be the main purpose of science 
teaching. However, unambiguous terminology is also important in scientific 
communication and — ultimately — for understanding. Some technical terms are 
therefore helpful for everyone, but the number of essential ones is relatively 
small. If teachers introduce technical terms only as needed to clarify thinking and 
promote effective communication, then students will gradually build a functional 
vocabulary. (AAAS, 1989, p. 190) 

Science, mathematics, and technology. The integration of science, mathematics, and 
technology has been a common expectation in current science education reform. Project 2061 
(AAAS, 1989) describes their interrelationships, as follows: 

Scientists see patterns in phenomena as making the world understandable; 
engineers also see them as making the world manipulable. Scientists seek to 
show that theories fit the data; mathematicians seek to show logical proof of 
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abstract connections; engineers seek to demonstrate that designs work. Scientists 
cannot provide answers to all questions; mathematicians cannot prove all possible 
connections; engineers cannot design solutions for all problems, (p. 25) 

The five sets of documents identify the role of mathematics in science learning in 
different ways (see Table 1). Project 2061 emphasizes mathematics as integral to science, 
whereas NSES does not include mathematics as a key component of science. The three 
assessment frameworks emphasize mathematics with varying degrees. Because mathematics is 
involved in all aspects of science learning, it is difficult to specify its role in science learning. In 
the framework presented here, two aspects of mathematics related to science are highlighted: (a) 
measurement concepts and (b) statistics and probability concepts. 

Although integration of mathematics and science has been a common expectation in 
science curriculum and instruction, research on the role of mathematics in science learning and 
the relationship between science and mathematics learning is limited. Because of this limited 
information, there is an increasing interest in examining the mathematical components of science 
learning. For example, the research, “Building bridges between mathematics and science,” 
attempts to integrate mathematical and scientific reasoning in elementary school (Lehrer & 
Schauble, 1996, 1998). Specifically, teachers develop ways to help students develop 
mathematical ideas, including spatial visualization, data, and measurements, that can serve as a 
bridge to scientific reasoning. 

All of the documents stress technology in science learning. Among many definitions of 
technology (Raizen, Sellwood, Todd, & Vickers, 1995), these documents consistently use 
technology to refer to engineering and design. NSES defines technology as design: 

As used in the Standards, the central distinguishing characteristic between science 
and technology is a difference in goal: The goal of science is to understand the 
natural world, and the goal of technology is to make modifications in the world to 
meet human needs. Technology as design is included in the Standards as parallel 
to science as inquiry, (p. 24) 

Project 2061 (AAAS, 1989) defines technology as engineering, design, or 
engineering design interchangeably: 

The component of technology most closely allied to scientific inquiry and 
mathematical modeling is engineering. In its broadest sense, engineering consists 
of construing a problem and designing a solution for it. (p. 25) 

Technology as engineering and design differs from “instructional technology,” 
including scientific tools, computers, and electronic devices (NRC, 1996, p. 24). 

Consistent with NSES, the framework proposed here does not include these tools and 
devices as part of technology; instead, they are considered as part of scientific 
investigation (to be described later). 
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Despite its importance in making science relevant and practical in everyday life, 
technology as engineering and design has been largely ignored in school science (Raizen et al., 
1995). However, the situation is changing. The current emphasis on technology is promoted by 
the Science-Technology-Society curriculum programs that focus on using science for the 
solution of natural and human-made problems in society (Bybee & DeBoer, 1994). There is a 
growing interest in engineering and design in science instruction. Design projects are relevant to 
students’ everyday experiences, are intuitively meaningful compared to abstract theoretical 
constructs, require applications of science concepts in solving complex tasks, and resemble 
problem solving in real-world situations. Design projects also require alternative solutions to 
complex problems and involve trade-offs of advantages and disadvantages with each alternative. 
Some examples include making a boat that maximizes the capacity of carrying a load for the 
concepts of flotation and buoyancy (Duschl & Petasis, 1995), designing an elbow as a type of 
lever (Penner, Giles, Lehrer, & Schauble, 1997), and using a model of heat flow in engineering, 
rather than kinetic molecular theory in physics (Linn & Muilenburg, 1996). 

Science in personal and social perspectives. This component is included in all the 
documents, except for NAEP. NSES and Project 2061 include a broad range of personal and 
social issues, whereas TIMSS and New Standards have a limited focus on a narrow range of 
specific issues. Main topics include health, population growth, natural resources, environmental 
issues, safety, and natural and man-induced hazards. 

Throughout the history of science education for the most part of this century, personal 
and social relevance of science has been considered important, in addition to science knowledge 
and inquiry (Atkin et al., 1997; Bybee & DeBoer, 1994). However, the relative importance of 
the personal and social dimension or the disciplinary dimension of science knowledge and 
inquiry has shifted at different times of science education reform. The current emphasis is 
promoted by the Science-Technology-Society curriculum programs. NSES and the other 
documents try to strike a balance in emphasis between these two dimensions. 

The current emphasis on science in personal and social perspectives is consistent with the 
vision of scientific literacy for all students to become educated citizens, rather than for a select 
few to become scientists. NSES states, “An important purpose of science education is to give 
students a means to understand and act on personal and social issues” (p. 107). The emphasis is 
beyond application of knowledge and skills into decision making about personal and social 
issues. As students investigate such issues, they become involved in exploring alternative, 
sometimes controversial, points of view that require moral and ethical considerations. The goal 
is to help students learn to make informed decisions about personal and social matters based on 
sound knowledge of science. 

The emphasis on personal and social perspectives of science can provide opportunities 
for students to appreciate both the contributions and limitations of science in dealing with social 
phenomena, to consider multiple perspectives in making personal decisions, to judge the impact 
and consequences of decisions, and to distinguish scientific knowledge and evidence from 
personal beliefs and opinions. Simple examples of science in personal and social perspectives 
include the choices between paper and plastic bags or between disposable and cloth diapers, each 
with benefits and negative impacts on production costs and the environment. Other examples 
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include the greenhouse effect, global warming, cloning, and nuclear energy, as well as many 
health and environmental issues. 

History and nature of science. All of the documents emphasize the history and nature of 
science. Although the two are closely related, each is described below. 

With regard to the history of science, the historical development of major discoveries in 
modem science is emphasized in all the documents. This historical development is defined in 
terms of the tradition of Western science since the Copemican revolution. The contributions of 
non-Westem cultures to science and technology are mentioned in NSES and Project 2061, but 
this issue is not included in the other documents. 

Both NSES and Project 2061 define Western science as the proper domain of science. 
Science for All Americans (AAAS, 1989) describes the history of science as follows: 

The recommendations in this chapter focus on the development of science, 
mathematics, and technology in Western cultures, but not on how that 
development drew on ideas from earlier Egyptian, Chinese, Greek, and Arabic 
cultures. The sciences accounted for in this book are largely part of a tradition of 
thought that happened to develop in Europe during the last 500 years — a tradition 
to which people from all cultures contribute today, (p. 136) 

NSES describes the contributions of all cultures to science and technology. These 
contributions are recognized in terms of technological inventions to solve human problems and 
needs, rather than the traditions of thought that define the nature and practice of science to 
understand and explain natural phenomena. NSES states historical perspectives in science as 
follows: 

Modem science began to evolve rapidly in Europe several hundred years ago. 

During the past two centuries it has contributed significantly to the 
industrialization of Western and non-Westem cultures. However, other, non- 
European cultures have developed scientific ideas and solved human problems 
through technology, (p. 201) 

In addition to major scientific discoveries in modem science and the contributions of all 
cultures, there are other important reasons for emphasizing historical perspectives in science. 
First, the historical accounts demonstrate science as a human endeavor, which is “influenced by 
societal, cultural, and personal beliefs and ways of viewing the world. Science is not separate 
from society, but rather science is a part of society” (NRC, 1996, p. 201). Second, the historical 
accounts also illustrate how the nature and practice of science has evolved over the past several 
hundred years (AAAS, 1989, pp. 135-153). Third, the evolution of scientific ideas in history 
provide insights into students’ naive or incorrect conceptions about natural phenomena, which is 
a common topic in research on conceptual change in science (see the review in AAAS, 1993). 
Finally, acknowledgments of contributions of all cultures in science and technology can be an 
incentive for students from diverse backgrounds, who have traditionally been under-represented 
in science and technology, to participate in these areas (Rodriguez, 1997a). 
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The nature of science involves several aspects, including the nature of scientific 
knowledge, the nature of scientific inquiry, and the scientific world view. The nature of scientific 
knowledge and inquiry is emphasized in all of the documents with varying degrees. NSES 
focuses more on inquiry; Project 2061 focuses more on scientific knowledge; and the three 
assessment frameworks include both. The scientific world view is stressed only in Project 2061 
(AAAS, 1989, 1993). 

The nature of scientific knowledge involves how the knowledge is generated, tested, 
disputed, and justified. Scientists across disciplines share a general agreement on what 
constitutes scientific knowledge (AAAS, 1989, pp. 5-13; NRC, 1996, pp. 171, 201). The 
credibility and power of a scientific theory depends on its ability to explain and predict a wide 
range of natural phenomena and show relationships among these phenomena. Scientific 
knowledge is subject to change and always under dispute and contention. Yet, most scientific 
knowledge has become stable and durable through numerous verifications across settings and 
over time. Even a new theory generally involves modifications, rather than complete rejection, of 
existing ideas. 

Scientists also share a general agreement on what constitutes a scientifically valid inquiry 
(AAAS, 1989, pp. 5-9; NRC, 1996, p. 23). Scientific inquiry involves a rigorous process of 
observation, experimentation, and validation, and generates explanations based on evidence, 
reasoning, and logic. Contrary to what is generally known as “the scientific method” with a set 
of procedures, there is no fixed set of steps to follow in scientific inquiry. 

The scientific world view indicates that science is a way of knowing that distinguishes 
itself from other ways of knowing and from other bodies of knowledge (AAAS, 1989, pp. 3-5; 
NRC, 1996, p. 201). Science seeks to understand how the world works through particular modes 
of observation, experimentation, and validation, described earlier. Project 2061 (AAAS, 1989) 
states, “There are many matters that cannot usefully be examined in a scientific way. There are, 
for instance, beliefs that — by their very nature — cannot be proved or disproved (such as the 
existence of supernatural powers and beings, or the true purposes of life)” (p. 4). NSES also 
states, “Explanations on how the natural world changes based on myths, personal beliefs, 
religious values, mystical inspiration, superstition, or authority may be personally useful and 
socially relevant, but they are not scientific” (201). 

Students’ understanding of the nature of scientific knowledge and inquiry has been 
investigated rather extensively (e.g., Ryan & Aikenhead, 1992; Songer & Linn, 1991). Because 
elementary students have not had enough experience to develop a notion of the nature of science 
and because it is difficult to gather and interpret their responses, most of the research has been 
conducted with middle and high school students. The students often have difficulty 
understanding the nature of scientific knowledge and inquiry. They see science as an 
accumulation of factual knowledge that is based solely on data and objective observations, but 
fail to consider the likelihood of biased observations and interpretations (Carey, 1985; Carey & 
Smith, 1993). Many also consider scientific knowledge as either absolutely true or always 
tentative and subject to change. In addition, they have difficulty differentiating between a theory 
and evidence, between description and interpretation of evidence, and between informed reasons 
and personal opinions (Kitchener & King, 1981; Kuhn, 1993, 1997). In general, students have 
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difficulty understanding science as a human endeavor to construct knowledge that is increasingly 
closer approximations to truth in understanding and explaining how the world works. 

The scientific world view is a topic that has recently become an important area of 
research. The research indicates that although the distinction between the scientific world view 
and alternative views may be relatively straightforward to educated adults, children’s world 
views involve a complex interaction of personal and supernatural beliefs with scientific 
understanding (Cobem, 1991; Hewson, 1988; Lee, in press; Loving, 1997). In addition to such 
developmental trends, the scientific world view is sometimes incompatible with the norms and 
values of diverse cultures which tend to include spiritual and supernatural forces (Hewson, 1988; 
Lee, in press). This presents greater challenges in developing the scientific world view for 
students from diverse cultures and languages than for mainstream students. 

Unifying themes. Unifying themes are emphasized in all the documents, except for 
TIMSS. Unifying themes indicate “big ideas” or “powerful ideas” that transcend a range of basic 
concepts and processes in science and technology (NSES) or in science, mathematics, and 
technology (Project 2061). Project 2061 states, “Some important themes pervade science, 
mathematics, and technology . . . They are ideas that transcend disciplinary boundaries and prove 
fhxitfiil in explanation, in theory, in observation, and in design” (AAAS, 1989, p. 155). Based on 
the NSES and Project 2061 documents, the framework here identifies five unifying themes, 
including systems, models, constancy and change, evolution and equilibrium, and form and 
function. 

Research on unifying themes has been conducted for some time. Studies are available in 
areas, including: (a) a system, sub-systems, and their relationships, e.g., the water cycle, air 
pressure, air masses, and fronts as sub-systems of the weather system; (b) the explanatory role of 
models for understanding and explaining natural phenomena, e.g., models of molecular and 
atomic structures, models of the solar system; (c) patterns of changes over time or across 
settings, e.g., changes in trends, cycles, or irregular patterns; (d) evolution and equilibrium, e.g., 
changes and constancy in ecology or species, and (e) form and function, e.g., forms and 
functions of biological organs, trade-offs between forms (beauty) and functions (utility) of 
design projects. 

Although the research has identified learning difficulties with unifying themes commonly 
experienced by students, there are limitations in the existing literature. One limitation is the 
restriction of research to specific tasks within a narrowly defined science area or a limited set of 
natural phenomena, rather than students’ understanding of unifying themes across science 
disciplines or a range of natural phenomena. Another limitation is the focus of research on 
separate aspects of student abilities, such as observation, explanation, theory, or design, rather 
than the combination of these abilities. For unifying themes to be truly big ideas, they need to be 
examined across disciplinary boundaries, a range of natural phenomena, and a spectrum of 
student abilities. 
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Science Process 



Science process indicates what students should be able to do with the components of 
science content. Science process is not independent of science content; instead, science process 
cuts across science content. The four components of science process includes: (a) scientific 
understanding; (b) scientific investigation; (c) scientific communication; and (d) habits of mind 
(see Table 2). 

Scientific understanding. Scientific understanding refers to the knowledge and 
understanding of key concepts and theories, and their applications to explain natural phenomena. 
This is consistent with “conceptual understanding” in New Standards,- “conceptual 
understanding” in NAEP, and “understanding” in TIMSS. NSES defines scientific knowledge 
and understanding as follows: 

Scientific knowledge refers to facts, concepts, principles, laws, theories, and 
models and can be used in many ways. Understanding science requires that an 
individual integrate a complex structure of many types of knowledge, including 
the ideas of science, relationships between ideas, reasons for these relationships, 
ways to use ideas to explain and predict other natural phenomena, and ways to 
apply them to many events, (p. 23) 

Based on the NSES definition and science education research (e.g., Kennedy, 1998; 
Secada, 1997), scientific understanding includes several distinct abilities: (a) acquisition of key 
concepts and theories in science disciplines, such as those specified in major reform documents 
(see Table 3); (b) construction of relationships between and among concepts and theories within 
a science discipline, for example, relationships between heat energy and changes of states of 
matter in physical science; (c) construction of relationships among concepts and theories across 
science disciplines, for example, molecular and atomic theories across physical changes, 
chemical changes, and human biological systems; (d) use of concepts and theories to explain and 
predict natural phenomena; and (e) applications of concepts and theories to new, real-world 
situations. 

Prior knowledge and personal experience play key roles in acquiring new knowledge and 
developing conceptual understanding (Driver et al., 1994; Posner, Strike, Hewson, & Gertzog, 
1982). Learning and understanding in science occurs when students successfully integrate new 
information with their prior experiences in ways that are both scientifically accurate and 
personally meaningful. Because students are always building new relationships among their 
ideas and restructuring those ideas, their understanding is dynamic, changing, and growing. 
Students’ understanding of science concepts has been studied extensively since the late 1970s 
(for review of the research, see chapter 15 in AAAS, 1993). Cognitive scientists or conceptual 
change researchers have examined students’ conceptions of natural phenomena using in-depth 
interviews. The research has revealed that students usually have ideas about how the world 
works based on common-sense experiences and that these misunderstandings or misconceptions 
remain highly resistant to change even after instruction. Despite this extensive body of literature. 
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only a small number of science concepts have been investigated, and the studies are unevenly 
distributed across science fields. 

Scientific investigation. The five sets of documents use two related terms, “scientific 
inquiry” and “scientific investigation” (see Table 1). NSES and Project 2061 use inquiry, 
whereas New Standards, 1996 NAEP, and TIMSS use investigation. The framework proposed 
here uses scientific investigation as it is used in New Standards, 1996 NAEP, and TIMSS. 
Distinctions between these two terms and meanings are made next. 

Both NSES and Project 2061 use scientific inquiry in a broad sense. NSES (NRC, 1996) 
provides a comprehensive definition: 

Scientific inquiry refers to the diverse ways in which scientists study the natural 
world and propose explanations based on the evidence derived from their work. 

Inquiry also refers to the activities of students in which they develop knowledge 
and understanding of scientific ideas, as well as an understanding of how 
scientists study the natural world, (p. 23) 

NSES provides an equally comprehensive description of what scientific inquiry 
involves: 

Inquiry is a multifaceted activity that involves making observations; posing 
questions; examining books and other sources of information to see what is 
already known; planning investigations; reviewing what is already known in light 
of experimental evidence; using tools to gather, analyze, and interpret data; 
proposing answers, explanations, and predictions; and communicating results. 

Inquiry requires identification of assumptions, use of critical and logical thinking, 
and consideration of alternative explanations, (p. 23) 

In this broad view, NSES emphasizes inquiry as an overarching goal of science education 
through which students develop the knowledge and abilities of other components of science 
achievement (NRC, 1996, p. 105). At the same time, NSES identifies inquiry as one component 
of science achievement, consisting of both “the ability to conduct inquiry” (process) and 
“understanding about scientific inquiry” (content) (p. 105). 

In contrast to NSES and Project 2061, New Standards, 1996 NAEP, and TIMSS use 
scientific investigation in a more limited and specific way. These documents consider scientific 
investigation as a systemic observation, a “fair” test (“a test in which only one variable at a time 
is changed” in NRC, 1996, p. 122), or a controlled experiment. During scientific investigation, 
students formulate questions, devise plans to explore the questions, make and revise hypotheses, 
collect and analyze data, interpret data, generate explanations, and draw conclusions. Students 
also use appropriate tools and equipment for conducting investigation. 

Scientific investigations occur at different developmental levels (AAAS, 1993; NRC, 
1996). Elementary students can do a systemic observation or a fair test. However, they have 
difficulty designing and conducting a carefully controlled experiment. They also have difficulty 
giving explanations based on evidentiary criteria, models, or cause and effect relationships. At 
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the middle school level, students become more systemic and sophisticated in conducting 
investigations. They develop an understanding of what constitutes a good experiment, although 
they have difficulty controlling variables systematically. They can also make distinctions 
between descriptions and explanations and provide explanations in terms of causality or models. 
High school students can conduct the entire process of scientific investigations, provide logical 
explanations based on evidence, make predictions about complex phenomena, and test 
alternative ideas and approaches. 

Scientific tools and equipment play important roles in scientific investigations. In early 
grades, students develop skills in using simple tools, such as rulers to measure the length and 
height of objects, scales and balances to measure weight and force, thermometers to measure 
temperature, watches to measure time, and magnifiers and microscopes to measure finer details 
of objects and organisms. As students gain experience, they learn to use tools and equipment for 
data collection, storage and retrieval of information, data analysis and pattern recognition, and 
organization and display of results. Computers and electronic devices provide rich and extensive 
data sources, such as access to real time data as natural phenomena and events unfold. 

Scientific communication. The roles of communication and discourse have been 
increasingly emphasized in science reform documents, as well as in science education research 
(Holliday, Yore, & Alverman, 1994; Lemke, 1990; Palincsar, Anderson, & David, 1993). 
Scientific communication is emphasized in New Standards and TIMSS. Although NSES and 
Project 2061 do not identify process standards, including scientific communication, the 
importance of scientific communication is stressed in the text of the documents. NSES (NRC, 
1996) emphasizes, “[Teachers] structure and facilitate ongoing formal and informal discussion 
based on a shared understanding of rules of scientific discourse. A fundamental aspect of a 
community of learners is communication” (p. 50). In the framework presented here, two aspects 
of scientific communication are highlighted: multiple representations and rules of scientific 
discourse. 

In the process of knowing and doing science, students develop and organize their 
knowledge and understanding through communication. They can demonstrate their knowledge 
and understanding using multiple representational formats, including oral communication, 
writing, drawings, charts, diagrams, tables, graphs, figures, and concept maps. Representational 
fluency also includes the use and interpretation of mathematical symbols and models, computer 
graphics and simulations, and spreadsheets. 

Children in the elementary grades may have difficulty with multiple representations. 

Until recently, much of the communication about science has been done through reading and 
writing, as opposed to oral communication (Yore, Holliday, & Alverman, 1994). With the 
growing emphasis on younger children and less literate students, speaking (Gallas, 1995) or 
using pictures and drawings (Lee & Fradd, 1996a) can be important means of communication. 
Because scientific discourse stresses explicit communication in written or symbolic forms, 
students gradually learn to present their understanding in concise and powerful representational 
formats. 
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As students engage in formal and informal discussion of science, they develop an 
understanding of the rules of scientific discourse (Anderson, Holland, & Palincsar, 1997; 
Palincsar, Anderson, & David, 1993). They learn how to explain, analyze, debate, justify, argue, 
defend, critique, and challenge the work of other students, as well as their own work. While 
engaging in scientific arguments, students use facts, evidence, logic, and reasoning to support or 
dispute conclusions (Rosebery, Warren, & Conant, 1992). Students gradually develop abilities to 
communicate clearly, construct reasoned arguments, and respond logically to critical comments. 

Scientific habits of mind. This component is emphasized in recent reform, especially in 
Project 2061. Project 2061 defines scientific habits of mind in terms of certain values, attitudes, 
and skills associated with science, mathematics, and technology (AAAS, 1989, chapter 12 and 
pp. 189-192). Although NSES uses the term “habits of mind” only once (p. 170) tluoughout its 
100 pages on content standards, the document emphasizes scientific values, attitudes, and skills 
throughout the text. Based on Project 2061 and NSES, scientific habits of mind in the 
framework here include: (a) scientific values and attitudes and (b) thinking skills. 

Of scientific values and attitudes (AAAS, 1989, chapter 12 and pp. 189-193; NRC, 1996, 
p. 50-5 1), some are generally shared in society and highly regarded as basic human qualities, - 
including curiosity, interest, insight, energy, diligence, persistence, intellectual honesty, and 
creativity. Others are central to science, including critical and independent thinking, tolerance of 
ambiguity or uncertainty, openness to new ideas, skepticism, empirical verification, arguments 
based on logic and evidence, and questioning rather than deferring to authority. In addition to 
independent thinking and performance, scientific values and attitudes also include teamwork, 
collaboration, and shared responsibility for learning with others in a science community. 
Scientific habits of mind also include thinking skills, such as heuristics of number sense and 
quantitative reasoning, logical skills, and metacognition in reflecting and self-assessing one’s 
own learning process and performance. 

The topic of scientific habits of mind is a new area of emphasis in recent reform. 
Attitudinal variables in science, such as interest, curiosity, and affect, have been studied 
extensively. But this research is generally based on students’ self-reports on questionnaires, 
rather than observations of students’ behavior or dispositions to provide more accurate and 
specific information. The scientific values and attitudes central to science, such as openness to 
new ideas, skepticism, questioning, and tolerance of ambiguity, have not been studied (for an 
exception, see Goldenberg, in press, in the case of mathematics education). These values and 
attitudes may not occur with young students until they have had sufficient experience in learning 
science in middle and high school years. Considering that scientific values and attitudes 
generally reflect and respond to the norms and practices of the western society in which modem 
science has evolved, these values and attitudes would pose greater difficulties to students from 
diverse cultures and languages than students from the mainstream (Lee & Fradd, 1996b). 

An Aggregated View of Science Achievement 

In summary, based on the analysis of commonalities and differences among major reform 
documents, an aggregated view of science achievement is described in this section and presented 
in Table 2. Several conclusions are drawn. First, science achievement is conceived in terms of 
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science content and science process, and the components of science process cut across the 
components of science content. Second, terms and meanings are sometimes used differently 
among these documents, and there is a need to develop a common language. Third, the 
components of science achievement overlap and relate to one another. Despite such an overlap, 
understanding the role that each component plays provides insight into its unique contributions 
as well as its interactions with other components. Finally, a definition of science achievement 
and identification of its components based on major reform documents provides a guideline for 
science assessment as well as science teaching and learning. 

Standards-Based Assessment in Large Education Systems 

At the heart of standards-based reform is the alignment of assessment with content 
standards. The National Academy of Education panel report states: ‘The intention of standards- 
based reform is to set higher standards for all students . . . New kinds of assessment reflecting 
these new standards are seen as instrumental in effecting the reform” (McLaughlin, Shepard, & 
O’Day, 1995, p. 52). Regardless of how challenging and rigorous the content standards are, if all 
the learning outcomes are not assessed, “teachers and students likely will redefine their 
expectations for learning science only to the outcomes that are assessed” (NRC, 1996, p. 82). In 
this section, implications of the analysis of science achievement for assessment in large 
education systems are described, particularly focusing on alignment of assessment with content 
standards. 

Specification of Content Standards into Assessment 

Aligning assessment with content standards is a complex task (Webb, 1997). The first 
step in standards-based assessment is to develop an assessment framework that is derived from 
content standards. The content standards should be stated as specific learning outcomes that can 
be measured. Then, based on the assessment framework, assessment specifications are designed 
— “specific aspects, limits, and boundary conditions” on the domains of knowledge and abilities 
to be assessed (Webb, 1997, p. 37). For 1996 NAEP, the assessment framework is described in 
NAGB (1996) and the assessment specifications in NAGB (1994). For TIMSS, the assessment 
framework is described in Robitallie et al. (1993) and the assessment specifications in McKnight, 
Schmidt and Raizen (1993). In New Standards, a set of performance descriptions for each 
performance standard serves as assessment specifications (NCEE, 1997a, 1997b, 1997c). 

Once assessment frameworks and specifications are designed, the next step in standards- 
based assessment is to develop assessment activities. A general concern with NSES is that the 
content standards are too broad and general to guide assessment activities. Even performance 
standards, such as New Standards, “are not specified well enough for purposes of test 
development. They do not adequately guide the concrete decisions that need to be made on what 
is to be measured, how it is to be measured, and what specific tasks and criteria will be used” 
(Wiley, cited in National Center for Research on Evaluation, Standards, and Student Testing, 
1997, p. 5). An extensive knowledge gap exists in specifying the standards to develop 
assessment activities (Massell, 1994; McLaughlin, Shepard, & O’Day, 1995). 
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Forms of Assessment 

To measure the kinds of knowledge and abilities expected in content standards, 
appropriate forms of assessment are required. There should be a match between what is to be 
measured and how best to measure it. Traditionally, large-scale assessments tend to focus on 
basic knowledge and skills and use restricted response forms, most commonly the multiple 
choice format (McLaughlin, Shepard, & O’Day, 1995). In contrast, higher-level thinking and 
complex abilities in science content standards require new forms of assessment (National Center 
for Research on Evaluation, Standards, and Testing, 1997). Although the increasing use of 
alternative assessments indicate current efforts, these assessments present new challenges, as 
described next. 

Large-scale assessments, such as NAEP and TIMSS, have changed significantly in recent 
years. Traditionally, NAEP science assessments used mostly multiple choice items with some 
open-ended items. The Second International Science Study [SISS] also used multiple choice 
items exclusively (International Association for the Evaluation of Educational Achievement, 
1988). In response to the current emphasis on scientific understanding, 1996 NAEP and TEMSS 
included open-ended, ffee-response items (including both short-answer and extended-response 
items) as well as multiple choice items. In addition, along with the emphasis on scientific 
investigation and communication, 1996 NAEP for the first time included performance exercises 
(also called hands-on tasks) (O’Sullivan, Reese, & Mazzeo, 1997, p. 42). TIMSS used 
performance tasks (also called hands-on activities) with a sub-sample of students. 

In performance exercises or tasks in 1996 NAEP and TEMSS, students manipulate 
materials, conduct scientific investigations, and communicate their observations and results. 
Despite its critical importance in standards-based assessment, performance measures in large- 
scale assessments, such as NAEP and TIMSS, present limitations. First, because of the need for 
standardization, performance exercises or tasks are provided for students along with the 
materials to use, procedures to follow, graphs or tables to report the data, and questions to 
answer. This standard procedure does not allow students to ask their own questions, design and 
conduct investigations, and communicate observations or results in their own ways. NAEP also 
cautions that performance tasks often become “follow-the-instructions” questions, rather than 
higher-level thinking in new contexts or applications to novel situations (NAGB, 1996, pp. 31- 
33). Second, performance exercises or tasks in 1996 NAEP and TIMSS are completed within 
limited time in one setting. This constraint does not allow scientific investigations of natural 
events as an on-going process. These limitations are inherent in external, large-scale 
assessments. 

New Standards focuses on performance assessments that teachers can use in the 
classroom, as well as states and urban school districts can use in systemic reform. As part of 
science instruction in the classroom. New Standards does not have the constraints of NAEP or 
TIMSS. The samples of student work included in the New Standards documents (NCEE, 1997a, 
1997b, 1997c) indicate that the assessment system effectively measures many components of 
science achievement. These components include scientific investigation and communication, as 
well as understanding of key concepts in three fields of science and unifying themes across the 
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fields. In contrast, the assessment system rarely addresses technology, science in personal and 
social perspectives, history and nature of science, and scientific habits of mind. 

The discussion here indicates inherent difficulties in the assessment of higher-level 
thinking and complex abilities in large-scale projects. First, some components of science 
achievement are difficult to assess. For example, it is difficult to operationalize abstract 
constructs, such as scientific habits of mind, in concrete and specific terms. Even after 
operational definitions are obtained, it is difficult to develop standardized procedures of 
assessment, for example, science in personal and social perspectives. There is a concern that in 
the process of making these components measurable, assessments may become trivialized and 
their importance become minimized. 

Second, certain components of science achievement can not be assessed on demand. For 
example, scientific investigation involves students’ asking their own question and finding 
answers on their own as an on-going process. Scientific habits of mind occur naturally as 
students engage in science tasks. It is difficult to assess students’ scientific investigation or 
habits of mind on demand within the confines of assessment settings. 

Finally, large-scale assessment generally involves written forms. Certain types of 
knowledge and abilities may require different forms of assessment. For example, assessment of 
technology (i.e., engineering and design) can be done by constructing actual products. 
Assessment of scientific habits of mind can be done informally as teachers observe cues of 
students’ dispositions (Webb, 1997, p. 22). Abilities to engage in scientific discourse can be 
observed as students engage in communication with others in group settings. Although a range 
of assessment forms can be utilized in classroom assessment, including observations, products, 
interviews, and portfolios, it is difficult to incorporate these forms in large-scale assessments. 

There is a clear indication that large-scale assessment projects, especially NAEP and 
TIMSS, influence state-level assessments (George & Van Home, 1996; Glaser & Linn, 1997; 
National Center for Research on Evaluation, Standards, and Student Testing, 1997). NAEP 
science assessments have been used for state-level results since 1990. States also have expressed 
an interest in creating linkages that allow the comparisons of state assessments with national and 
state-level NAEP (Glaser & Linn, 1997). Some states incorporate released items from NAEP 
and TIMSS in their assessment programs, compare their achievement results with those of other 
states or countries, and analyze their science curricular and teaching practices (Champagne, 
1997). As systemic reform continues and is likely to intensify (American Federation of 
Teachers, 1997; Council of Chief State School Officers, 1997; McLaughlin, Shepard, & O’Day, 
1995), the central role of assessment in evaluating the impact of standards-based and systemic 
reform on student achievement will increase at national and state levels. 

Equity in Large Education Systems 

Educational equity is emphasized, along with high academic standards, in systemic 
reform. The focus on equity is an attempt to address significant achievement gaps among 
students from diverse backgrounds in terms of ethnicity, language, gender, disabilities, and 
socio-economic levels. Standards-setting is an important first step in achieving equity because 
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educators now know the expectations for all students. Without resources and opportunities, 
however, setting high academic standards may pose additional challenges and learning 
difficulties to these students (Kahle, 1997; Porter, 1995). There is a great concern that “lack of 
support in reaching high standards will further victimize students already harmed by gross 
inequities in the educational system” (McLaughlin, Shepard, & O’Day, 1995, p. 68). 

To start the discussion on equity, its meaning needs to be considered. Equity is defined in 
many different ways, and these definitions are often inconsistent and even contradictory (Lynch 
et al., 1996; Secada, 1994). Within the scope of this paper on science achievement, the 
discussion here is limited to equity issues in relation to content standards and standards-based 
assessment in large education systems. As the discussion indicates, implications of the analysis 
of science achievement for equity present tensions and dilemmas, but the existing research and 
literature is limited and insufficient to resolve these tensions and dilemmas. 

Equity in Relation to Content Standards 

Both NSES and Project 2061 emphasize equity along with excellence as a dual goal of 
science education reform. NSES (NRC, 1996) highlights equity as the first of its four guiding 
principles: ""Science is for all students. This principle is one of equity and excellence” (p. 20, 
original emphasis). The premise of Project 2061 is equity, as reflected in the title of the 
document. Science for All Americans (1989). What counts as science and what should be taught 
in school science as presented in these documents, however, are often incompatible with ways of 
knowing and thinking by diverse students. These issues are addressed next. 

What counts as science? Equity concerns in science achievement begin with the basic 
notion of “what counts as science?” and “what should be taught in school science?” (Lee, 1997). 
As described about the history and nature of science in the previous section, NSES and Project 
2061 define Western science as the proper domain of science. NSES recognizes the 
contributions of diverse cultures for technological inventions, but not in terms of the tradition of 
science to understand and explain natural phenomena. 

Recently, alternative views of science have been advocated by scholars in emerging areas 
of multicultural education, feminism, and sociology and philosophy of science (Atwater & Riley, 
1993; Eisenhart, Finkel, & Marion, 1996; Hodson, 1993; Matthews, 1994; Rodriguez, 1997a; 
Stanley & Brickhouse, 1994). These scholars raise issues of power and marginalization of non- 
Westem groups, and challenge the basic notion of science and science achievement as 
traditionally defined. They highlight scientific and technological traditions of non-Westem 
cultures and argue for more inclusive views of science. 

As this debate suggests, what counts as science raises a serious question about equity in 
science achievement. The Western view of science as currently practiced in the science 
community and taught in school science presents “high status knowledge,” and every student 
should have access to such knowledge. On the one hand, the emphasis on the high status 
knowledge without consideration of diverse views about the nature of science may make science 
less accessible, relevant, or meaningful for some students, particularly those who have 
traditionally been bypassed in science education. On the other hand, the emphasis on diverse 



34 




43 



views that are culturally significant but marginally important as science topics in the science 
community and in school science may not promote equitable outcomes. There needs to be 
recognition of the views and contributions of diverse cultures in science and technology in 
defining what counts as science and what should be taught in school science. This tension 
remains unresolved. 

fVays of knowing and thinking by diverse students. In considering equity in science 
achievement, it is important to examine the extent to which the nature of science is compatible or 
incompatible with the background knowledge and experiences of students from diverse 
backgrounds (Atwater, 1994; Lee & Fradd, 1998; Lee, Fradd, & Sutman, 1995). The emerging, 
although limited, body of research indicates that students from diverse backgrounds bring with 
them their own ways of looking at the world that are representative of their environments and 
personal experiences. Their ways of knowing and thinking may or may not be compatible with 
the nature of science or the way science is generally taught in science class. A few examples, 
below, illustrate the differences between the nature of science as defined in content standards and 
diverse views of science. 

For example, the emphasis on “scientific inquiry into authentic questions generated from 
student experiences” (NRC, 1996, p. 3 1) may pose challenges for students from cultures 
emphasizing teachers’ authority of telling and directing students, rather than promoting students’ 
exploration or alternative solutions (Atwater, 1994; Lee & Fradd, 1998). Because inquiry is not 
part of their cultural experiences, the students need to be explicitly taught how to do inquiry as 
they learn to ask their own questions and find answers on their own. As the students develop the 
abilities to investigate and explore in the science classroom, however, they may recognize 
conflicts between their school experiences and home expectations for authority. The tension 
between science knowledge in school and everyday knowledge at home can present learning 
difficulties to these students. 

Cultivation of scientific habits of mind can also pose difficulties to students from diverse 
cultures and languages. Although some scientific values and attitudes are found in most cultures, 
others are characteristic of Western science, such as thinking critically and independently, using 
empirical criteria, making arguments based on evidence and logic, questioning, skepticism, 
openly criticizing, and tolerating ambiguity. These values and attitudes may be incongruent with 
the norms of diverse cultures that favor cooperation, social and emotional support, and consensus 
building. Enabling these students to acquire scientific values and attitudes, while retaining their 
own cultural norms, requires careful consideration. Using the notion of “border crossing,” 
students have more difficulties crossing the cultural boundaries between their everyday world 
and the world of science when the discrepancies are greater (Aikenhead, 1996; O’Loughlin, 
1992). Faced with challenges in border crossing, some students from diverse backgrounds may 
become alienated from science or even actively resist learning science. These students develop 
habits of mind that are opposite of scientific habits of mind. 

This discussion about differences between the nature of science as presented in content 
standards and various ways of knowing and thinking with diverse students raises an important 
question about equity in science achievement (Atwater, 1994; Lee & Fradd, 1998). On the one 
hand, in order to promote science achievement, it is important to relate the nature of science to 
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the students’ background knowledge and experience based on their culture, language, gender, 
and abilities. On the other hand, by providing information for each specific group, there is a 
danger of creating stereotypes for the group and also masking generalities across groups. 

Equity in Relation to Standards-Based Assessment 

Equity consideration in assessment begins and ends with achievement results among 
diverse groups of students. In the U.S., there are achievement gaps among ethnic, socio- 
economic, and gender groups (e.g.. National Center for Education Statistics, 1992; National 
Science Foundation, 1994; O’Sullivan, Reese, & Mazzeo, 1997). Beyond this general 
information, there is limited information about disaggregation of achievement results for gender- 
by-ethnic groups, or sub-groups within an ethnic group (Rodriguez, 1997b). 

There is also limited information about specific populations, such as students with 
disabilities and limited English proficient students because these students are often exempted 
from state and district assessments used for accountability (August & Hakuta, 1997; 

McLaughlin, Shepard, O’Day, 1995). Recently, large-scale assessments tend to include more 
students with disabilities and limited English proficient students (George & Van Home, 1996; 
Glaser & Linn, 1997). For example, on the basis of the inclusion criteria to assess “the 
achievement of all students at a given grade or age,” 1996 NAEP included students with 
disabilities and limited English proficient students (O’Sullivan, Reese, & Mazzeo, 1997, p. 55, 
original emphasis). 

When students with special needs participate in assessments, accommodations are made 
to ensure that the students can demonstrate their knowledge and abilities accurately. For 
example, 1996 NAEP offered various assessment accommodations for students with disabilities 
and limited English proficient students. These accommodations included one-on-one testing, 
small group testing, extended time, oral reading of directions, signing of directions, use of 
magnifying equipment, use of an individual to record answers, enlarged versions of test booklets, 
a Spanish/English glossary of science terms, and bilingual dictionaries (O’Sullivan, Reese, & 
Mazzeo, 1997). With certain accommodations (e.g., dictate responses), students with learning 
disabilities outscored regular students in state-wide performance assessment (Koretz, 1997). 
Assessment accommodations, however, raise questions about the validity of achievement results 
and the comparability of these results to results obtained under the condition without 
accommodations. 

Although limited English proficient students are more likely to be assessed than students 
with disabilities, limited English proficient students are less likely to be given accommodations 
(National Center for Research on Evaluation, Standards, and Testing, 1997). Assessments for 
limited English proficient students need to be done in the language of instruction, with special 
assistance in their first language (August & Hakuta, 1997; Shaw, 1997). This accommodation 
raises a question about the validity of results (Glaser & Linn, 1997). Even on bilingual versions, 
it would be difficult to distinguish the students’ science achievement Irom their English language 
proficiency and general literacy in the first language (McLaughlin, Shepard, & O’Day, 1995; 
Ruiz-Primo & Shavelson, 1996). 
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Science assessment between standardized forms and alternative forms of assessment has 
been a topic of debate. Critics of standardized tests charge that these tests are biased in terms of 
ethnicity, gender, and social class. They argue that standardized tests generally reflect the 
mainstream culture, contain content bias, incorporate linguistic and cultural bias, and fail to 
include diverse student populations in the norming process (Darling-Hammond, 1994; Garcia & 
Pearson, 1994; Supovitz & Brennan, 1997). In an effort to promote equity in assessment, many 
educators advocate alternative forms of assessment (Darling-Hammond, 1994; Garcia & 

Pearson, 1994). They claim that alternative assessments provide diverse students with flexible 
and multiple types of assessment settings, allow the students to participate in assessment 
activities to be consistent with their cultural preferences, and enable the students to communicate 
ideas in multiple ways that may not occur in a particular standard format. The relative equity of 
standardized tests and alternative assessments, however, indicates mixed results with ethnic, 
socio-economic, and gender groups (see Supovitz & Brennan, 1997 in language arts). In 
addition, the more flexible and varied assessment activities and settings are, the more difficult it 
becomes to incorporate them in large-scale assessments. 

Science assessment based on rigorous content standards presents both promises and fears 
with diverse student groups (see the discussion in Lynch et al., 1996; McLaughlin; Shepard, & 
O’Day, 1995; Ruiz-Primo & Shavelson, 1996; Shaw, 1997). Proponents claim that achievement 
gaps among ethnic, socio-economic, and gender groups may narrow because standards-based 
assessments focus on meaning and relevance, rather than discrete knowledge from textbooks. 
They suggest that authentic tasks drawn from the students’ real-life situations motivate and 
enhance their performance. Skeptics, on the other hand, argue that standards-based assessment 
may widen the achievement gaps because open-ended tasks and application of knowledge to 
novel situations may differentially favor students with many opportunities in science-rich 
environments of home and community over those lacking such opportunities. The skeptics 
suggest that assessment tasks based on the content and experiences within the classroom are 
more fair than those requiring knowledge and abilities in novel situations. Empirical evidence to 
test each of the two positions is limited. 

Conclusions and Discussion 

This paper reviews and analyzes the conceptions of science achievement in major reform 
documents, including those on content standards (NSES and Project 2061), performance 
standards (New Standards), and large-scale assessment frameworks (1996 NAEP and TIMSS). 
Analysis of commonalities and differences in the conceptions of science achievement in these 
documents leads to two key conclusions. First, there is an overall agreement on the conceptions 
of science achievement among the documents. The documents define science achievement in a 
comprehensive manner, including concepts and theories in physical, life, and earth and space 
science; scientific inquiry or investigation; science with mathematics and technology; science in 
personal and social perspectives; nature and history of science; unifying concepts or common 
themes; and scientific habits of mind. Second, there are some notable differences in the 
conceptions of science achievement among the documents. These differences are partially due to 
different contexts and purposes of the documents. 
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Based on the synthesis of the conceptions of science achievement in these documents, the 
paper presents an aggregated view of science achievement. Science achievement is conceived in 
terms of science content and process. The components of science content include: (a) concepts 
and theories in physical, life, and earth and space science; (b) science, mathematics, and 
technology; (c) science in personal and social perspectives; (d) history and nature of science; and 
(e) unifying themes. The components of science process include: (a) scientific understanding; 

(b) scientific investigation; (c) scientific communication; and (d) scientific habits of mind. The 
components of science process cut across and intersect with the components of science content. 

The paper considers the implications of the aggregated view of science achievement for 
large-scale assessments. The alignment of assessment with content standards is an essential, but 
difficult and complex, task. Specifying the content standards into assessment frameworks and 
assessment activities is not straightforward. To measure higher-level thinking and complex 
abilities emphasized in the content standards, new forms of assessment are required. In large- 
scale assessments, some components of science achievement present challenges because it is 
difficult to operationalize them in concrete terms, to develop standardized procedures, to 
administer on-demand assessment, or to use multiple forms of assessment (e.g., observations, 
interviews, products) in addition to written forms. As a result, these components of science 
achievement may be left out in assessments. 

New Standards, 1996 NAEP, and TIMSS present significant innovations in large-scale 
assessments. All three projects developed assessment frameworks that were generally aligned 
with science content standards. They also used alternative forms of assessment that broke from 
the restricted multiple choice format traditionally used in large-scale assessments. The 1996 
NAEP and TIMSS used performance tasks and open-ended response items as well as multiple 
choice items. New Standards focuses specifically on performance assessment . Despite such 
innovations, these projects also indicate challenges and limitations in assessing high academic 
standards in large-scale assessments. The innovations and limitations will provide valuable 
information for developing large-scale assessments in standards-based and systemic reform 
(Suter, 1994). 

Finally, the paper considers the implications of the aggregated view of science 
achievement for equity. Although major reform documents emphasize educational equity, there 
are tensions and dilemmas in considering equity related to science content standards and 
standards-based assessment. In defining what counts as science and what should be taught in 
school science, there is a tension between the Western view of science as represented in NSES 
and Project 2061 and views of science in diverse cultures. The nature of science in the Western 
view is often incompatible with ways of knowing and thinking in diverse cultures. In addition, 
the relative equity of standardized forms and alternative forms of assessment is under 
consideration. The equity in assessing rigorous content standards beyond the background 
knowledge and experiences of students who have limited science opportunities is also under 
consideration. 

In closure, major reform documents in science education consistently emphasize high 
achievement for all students. The available knowledge about assessment and equity, however, is 
limited. The dilficulties with large-scale assessments are conceptual and practical, in terms of 
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how to do the assessment within the confines of assessment settings. The difficulties with 
educational equity are partially ideological and cultural, in terms whose science should count and 
be taught in school science, in addition to practical matters of resources and opportunities. Now 
that science content standards are established, efforts should be focused on how to implement 
standards-based assessments and how to ensure access and achievement for all students. The 
alignment of assessment with the content standards, as well as the attainment of the standards by 
all students, are key challenges to standards-based and systemic reform in large education 
systems. 
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